Skip to content

feat(orchestrator)!: finalize batches from their speculation paths - #507

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-runfrom
preetam/speculation-finalization
Open

feat(orchestrator)!: finalize batches from their speculation paths#507
behinddwalls wants to merge 1 commit into
preetam/speculation-runfrom
preetam/speculation-finalization

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Speculation was building the right things and merging on the wrong rule. A head still waited for every dependency to succeed before it could merge — the same rule as before any of this — so a batch built without a slow neighbour sat behind that neighbour anyway. The paths were being earned and then ignored. This is the commit that collects them.

What?

A head merges once one of its passed paths has had every dependency it assumed would succeed actually merge; assumed-failing and ignored dependencies impose no wait. That is the whole speedup: the head waits on what its passing build was actually stacked on, not on its full dependency list. A head fails only when no funded path has a future left, and the winner's siblings are superseded — cancelled to free their slots.

Outcomes are reached in finalize, before the Speculator is asked, and committed one generation at a time, so a cascade (A fails → B's last path breaks → B fails → maybe C too) resolves in a single run without ever enacting a dependent of an outcome whose own write lost its compare-and-swap. A batch decided by a cascade gets a recovery signal before it turns terminal, since no retry of the triggering message would ever revisit it.

Cancellation joins the run: a cancelling batch is just another batch the run walks — its live paths are marked cancelling, the poll loop stops their builds, and whichever later run sees them all stopped drives the batch to Cancelled. A cancelling path with no build link is cancelled immediately; there is deliberately no reservation state and no staleness bound, so a crashed dispatch can never keep a batch out of its terminal state.

The legacy per-batch finalizer is deleted; speculate.go keeps only admission and message routing, and the package doc gains the batch lifecycle, a worked example, and the finalize step.

Test Plan

bazel test //submitqueue/orchestrator/... — the merge rule is table-driven per assumption; failure, cascade commit ordering, and CAS-loss isolation are covered, and cancellation end to end: never-dispatched, live-build, no-paths, and lost-race cases.

make fmt, make gazelle

Stack

  1. docs(rfc): explain best-first speculation generation #513
  2. feat(speculation): generator contract and bestfirst impl #446
  3. feat(speculation): allocator contract and sticky impl #450
  4. feat(speculation): standard composed speculator #451
  5. feat(storage): speculation path set store #501
  6. feat(storage): path-build link store for per-path builds #502
  7. feat(orchestrator): dispatch builds per speculation path #503
  8. feat(orchestrator): poll builds and stop the ones nothing wants #504
  9. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  10. feat(orchestrator): re-plan the queue from the Speculator each run #506
  11. @ feat(orchestrator)!: finalize batches from their speculation paths #507
  12. feat(orchestrator): compose per-queue speculators and turn speculation on #508

@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from dfb687a to 2b97a64 Compare August 4, 2026 04:46
@behinddwalls
behinddwalls marked this pull request as ready for review August 4, 2026 04:47
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code owners August 4, 2026 04:47
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 2b97a64 to 93573ae Compare August 4, 2026 05:11
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 93573ae to 1492c63 Compare August 5, 2026 03:15
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 1492c63 to 23b4741 Compare August 5, 2026 03:26
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 23b4741 to 88bb6b3 Compare August 5, 2026 03:38
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 88bb6b3 to d9beb4d Compare August 5, 2026 03:44
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from d9beb4d to 42b8e81 Compare August 5, 2026 04:12
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 42b8e81 to 26ae2d9 Compare August 5, 2026 04:57
## Summary

### Why?

Speculation was building the right things and merging on the wrong rule. A head still waited for *every* dependency to succeed before it could merge — the same rule as before any of this — so a batch built without a slow neighbour sat behind that neighbour anyway. The paths were being earned and then ignored. This is the commit that collects them.

### What?

A head merges once one of its **passed** paths has had every dependency it *assumed would succeed* actually merge; assumed-failing and ignored dependencies impose no wait. That is the whole speedup: the head waits on what its passing build was actually stacked on, not on its full dependency list. A head fails only when no funded path has a future left, and the winner's siblings are superseded — cancelled to free their slots.

Outcomes are reached in `finalize`, before the Speculator is asked, and committed one generation at a time, so a cascade (A fails → B's last path breaks → B fails → maybe C too) resolves in a single run without ever enacting a dependent of an outcome whose own write lost its compare-and-swap. A batch decided by a cascade gets a recovery signal before it turns terminal, since no retry of the triggering message would ever revisit it.

Cancellation joins the run: a cancelling batch is just another batch the run walks — its live paths are marked cancelling, the poll loop stops their builds, and whichever later run sees them all stopped drives the batch to Cancelled. A cancelling path with no build link is cancelled immediately; there is deliberately no reservation state and no staleness bound, so a crashed dispatch can never keep a batch out of its terminal state.

The legacy per-batch finalizer is deleted; `speculate.go` keeps only admission and message routing, and the package doc gains the batch lifecycle, a worked example, and the finalize step.

## Test Plan

✅ `bazel test //submitqueue/orchestrator/...` — the merge rule is table-driven per assumption; failure, cascade commit ordering, and CAS-loss isolation are covered, and cancellation end to end: never-dispatched, live-build, no-paths, and lost-race cases.

✅ `make fmt`, `make gazelle`

# Conflicts:
#	submitqueue/orchestrator/controller/speculate/BUILD.bazel
#	submitqueue/orchestrator/controller/speculate/speculate_test.go

# Please enter the commit message for your changes. Lines starting
# with '#' will be kept; you may remove them yourself if you want to.
# An empty message aborts the commit.
#
# interactive rebase in progress; onto ec05b712
# Last command done (1 command done):
#    pick 4462c89 # feat(orchestrator)!: finalize batches from their speculation paths
# No commands remaining.
# You are currently rebasing branch 'preetam/speculation-finalization' on 'ec05b712'.
#
# Changes to be committed:
#	modified:   submitqueue/orchestrator/controller/cancel/cancel.go
#	modified:   submitqueue/orchestrator/controller/speculate/BUILD.bazel
#	modified:   submitqueue/orchestrator/controller/speculate/check.go
#	modified:   submitqueue/orchestrator/controller/speculate/dispatch.go
#	modified:   submitqueue/orchestrator/controller/speculate/doc.go
#	new file:   submitqueue/orchestrator/controller/speculate/finalize.go
#	new file:   submitqueue/orchestrator/controller/speculate/outcome.go
#	new file:   submitqueue/orchestrator/controller/speculate/outcome_test.go
#	modified:   submitqueue/orchestrator/controller/speculate/run.go
#	modified:   submitqueue/orchestrator/controller/speculate/run_test.go
#	modified:   submitqueue/orchestrator/controller/speculate/snapshot.go
#	modified:   submitqueue/orchestrator/controller/speculate/speculate.go
#	modified:   submitqueue/orchestrator/controller/speculate/speculate_test.go
#

# Conflicts:
#	submitqueue/orchestrator/controller/speculate/speculate.go

# Please enter the commit message for your changes. Lines starting
# with '#' will be kept; you may remove them yourself if you want to.
# An empty message aborts the commit.
#
# interactive rebase in progress; onto 9a490fe
# Last command done (1 command done):
#    pick ec12146f # feat(orchestrator)!: finalize batches from their speculation paths
# No commands remaining.
# You are currently rebasing branch 'preetam/speculation-finalization' on '9a490fe4'.
#
# Changes to be committed:
#	modified:   submitqueue/orchestrator/controller/speculate/BUILD.bazel
#	modified:   submitqueue/orchestrator/controller/speculate/check.go
#	modified:   submitqueue/orchestrator/controller/speculate/dispatch.go
#	modified:   submitqueue/orchestrator/controller/speculate/doc.go
#	new file:   submitqueue/orchestrator/controller/speculate/finalize.go
#	new file:   submitqueue/orchestrator/controller/speculate/outcome.go
#	new file:   submitqueue/orchestrator/controller/speculate/outcome_test.go
#	modified:   submitqueue/orchestrator/controller/speculate/run.go
#	modified:   submitqueue/orchestrator/controller/speculate/run_test.go
#	modified:   submitqueue/orchestrator/controller/speculate/snapshot.go
#	modified:   submitqueue/orchestrator/controller/speculate/speculate.go
#	modified:   submitqueue/orchestrator/controller/speculate/speculate_test.go
#
@behinddwalls
behinddwalls force-pushed the preetam/speculation-finalization branch from 7737f77 to 7fd59cb Compare August 5, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant